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APPEAL BRIEF 



Sir: 



This brief is in support of the June 12, 2007 notice of appeal of the final 
rejection of the claims. 

(i) Real Party in Interest 

Siemens Corporate Research, Inc., Princeton, N.J. 

(ii) Related Appeals and Interferences 

None. 

(iii) Status of Claims 

Claims 31-40 are pending in this application. Claims 1-30 have been 
canceled. Claims 31-40 stand rejected. 

The rejection of claims 31-40 are appealed. 
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(iv) Status of Amendments 

A request for reconsideration after final rejection, submitted on May 3, 
2007 was entered by the Examiner. 

(^) Summary of the Claimed Subject Matter 

An augmented reality system allows a user to determine the depth of an 
instrument being inserted into an object from an external perspective. An 
augmented reality view is presented by overlaying a virtual graphics guide onto a 
real view of the object and an instrument. The graphics guide comprises a virtual 
depth marker located outside of the object. The instrument is aligned to the 
graphics guide. The instrument is inserted to a depth determined in the 
augmented view by alignment of a predetermined feature of the instrument with 
the virtual depth marker. The feature is located along the length of the 
instrument at a certain distance from the instrument tip and remains external to 
the object during insertion. 

(vi) Grounds of rejection to be reviewed on appeal 

(1) Obviousness of claims 31-40 as unpatentable over an article 
entitled 'The Expert Surgical Assistant: An Intelligent Virtual 
Environment with Multimodal Input" (Billinghurst) in view of U.S. 
Patent No. 6,470,207 (Simon). 

(vii) Argument 

Billinghurst and Simon Do Not Render 
The Claims Obvious as They Do Not Teach 
All of the Claimed Elements 

Claims 31-40 stand rejected under 35 U.S. C. §103 (a) as obvious in view 
of an article entitled 'The Expert Surgical Assistant: An Intelligent Virtual 
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Environment with Multimodal Input" (Billinghurst) in view of U.S. Patent No. 
6,470,207 (Simon). 'To establish prima facie obviousness of a claimed 
invention, all the claim limitations must be taught or suggested by the prior art." 
M.P.E.P. §2143.03 (8* Ed., rev. 4, October 2005), p. 2100-139, citing In re 
Royka, 490 F.2d, 981, 180 U.S. P.O. 580 (C.C.P.A. 1974) [emphasis in original]. 
The cited references fail to teach all of the limitations in the claims. 

Billinghurst discloses a system for addressing instrument location and 
navigation that tracks the location of the instrument through images. Billinghurst 
does not teach or disclose "inserting the real instrument to a depth determined in 
the augmented view by alignment of a predetermined real feature of the real 
instrument with the virtual depth marker, the real feature being located along the 
length of the real instrument at a certain distance from the real instrument tip, 
and remains external to the real object during insertion" as recited in independent 
claim 31 . Nor does Billinghurst teach or disclose "a rendering device for 
rendering the virtual graphics guide onto a real view of the real object and a real 
instrument such that the real instrument can be inserted to a depth determined in 
the augmented view by alignment of a predetermined real feature of the real 
instrument with the virtual depth marker, the real feature being located along the 
length of the real instrument at a certain distance from the real instrument tip, 
and remains external to the real object during insertion" as recited in independent 
claim 36. 

In the analysis of claims 31 and 36, the office action recites "the graphics 
guide comprising a virtual depth marker located outside of the real object {figure 
2,0, pp. 596)". The referenced image shows a trajectory for instrument 
movement based on a defined starting point. There is no virtual depth marker 
external to the object and Billinghurst does not teach or disclose a real 
instrument having a real feature that is aligned with the virtual depth marker and 
which remains external to the real object (i.e., anatomical structure). The 
instrument tip is marked on the images in Billinghurst with a cross which moves 
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according to the user's Instrument motion. The CT scan also changes in 
response to instrument depth, so the current images are those corresponding to 
the instrument tip location within the nasal cavity, Billinghurst uses this tracking 
mechanism to monitor the position of the instrument. 

Simon discloses a surgical navigational guidance system which uses one 
or more fluoroscopic x-ray images. Simon uses a virtual cone as a mathematical 
reference to the volume that is being imaged by the X-ray imaging system. The 
location and opening angle of the virtual cone determine the field-of-view of the 
X-ray system. Simon determines the coordinates of the virtual cone with respect 
to the coordinate system of the tracking system so that he can overlay a virtual 
model of the tracked instrument spatiaify correct onto the X-ray image of the 
patient's anatomy. Tracking of the instrument is a prerequisite for Simon's 
method. 

Like Billinghurst, Simon does not teach or disclose the claim elements 
recited in independent claims 31 and 36. The virtual cone is not shown to the 
user and therefore cannot provide any guidance to the user; in particular, it does 
not serve as a virtual depth marker. As indicated in column 12, lines 10-23, the 
virtual cone is known in the coordinate system of the tracking system. The cone 
is used to understand the spatial relationship of the X-ray system's field-of-view 
with the tracked position of the instrument, which allows one to indicate the 
instrument position (known to the system by instrument tracking) in the 
fluoroscopic image with a virtual instrument. The virtual instrument can be used 
by the physician to guide the trajectory of the real instrument. Simon does not 
align a real feature of the instrument with the virtual marker in order to insert the 
instrument to a predetermined depth. 

The office action also refers to column 7, lines 49-51 where Simon 
discloses an instrument embedded with infrared emitters or reflectors. The 
infrared emitters or reflectors are different from the real feature recited in the 
claims because they do not remain external to the object and the infrared 



4 



Appl. No. 09/818,123 

Appeal Brief dated August 8, 2007 

Reply to Office Action of March 14, 2007 

emitters are used with a tracking sensor (see col. 7, lines 45-61) to communicate 
to a computer the location of the instrument. The emitters would be located on 
the part of the instrument that is inserted into the body to assist in tracking the 
movement of the instrument within the body. 

The combination of Billinghurst and Simon lack all of the claimed elements 
and thus the claims are not obvious in view of this art. Since both Billinghurst 
and Simon fail to teach, disclose or suggest aligning a virtual depth marker with a 
real feature of a real instrument such that the real feature remains external to the 
real object during insertion. Claims 32-35 and 37-40, being dependent upon 
independent claims 31 and 36 respectively, are also not taught, disclosed or 
suggested by the combination of Billinghurst and Simon. The combination 
suggested here is deficient and ineffective as the references, individually and in 
combination, fail to teach all of the claimed elements. Additionally, there is no 
teaching or suggestion in any of the references to modify them in such a manner 
that would result in the claimed invention. 

Conclusion 

Since the references do not teach, suggest or disclose all of the claimed 
elements, the rejection under §103 cannot stand and the claims therefore are 
distinguishable over the cited art. The applicants respectfully request that the 



5 



Appl. No. 09/818,123 

Appeal Brief dated August 8, 2007 

Reply to Office Action of March 14, 2007 



Board reverses the examiner and direct that the application be passed to 
allowance. 



Respectfully submitted, 




Michele L. Conover 
Reg. No. 34,962 
Attorney for Applicant 



Date: August 8. 2007 
Siemens Corporation 
Intellectual Property Department 
170 Wood Avenue South 
Iselin, New Jersey 08830 
(732) 321-3013 
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(viii) Ciaims Appendix 

31 . A metliod for augmented reality guided positioning of a real instrument tip 
within a real target located in a real object comprising the steps of: 

presenting an augmented reality view by overlaying a virtual graphics 
guide onto a real view of the real object and a real instrument, the graphics guide 
comprising a virtual depth marker located outside of the real object; 

aligning the real instrument to the graphics guide; 

inserting the real instrument to a depth determined in the augmented view 
by alignment of a predetermined real feature of the real instrument with the 
virtual depth marker, the real feature being located along the length of the real 
instrument at a certain distance from the real instrument tip, and remains external 
to the real object during insertion. 

32. The method of claim 31 wherein the presenting step further includes 
interactive planning which comprises the steps of: 

determining an optimal location for the real instrument with respect to the 
real target; 

calculating the proximity of the predetermined portion of the real 
instrument to the real target based on the optimal location and the geometry of 
the real instrument; 

using the proximity calculation to determine the position of the virtual 
depth marker on the virtual graphics guide. 

33. The method of claim 32 wherein the proximity calculation comprises a 
range a proximity measurements. 
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34. The method of claim 32 wherein the proximity calcuiation corresponds to a 
final forward position of the predetemiined portion of the real instrument with 
respect to the real target. 

35. The method of claim 32 wherein the proximity calculation corresponds to a 
distance between the virtual depth marker and a point within the real target. 

36. Apparatus for augmented reality guided instrument positioning of a real 
instrument tip within a real target located in an real object comprising: 

a virtual graphics guide generator and positioner for generating and 
positioning a virtual graphics guide, the graphics guide comprising a virtual depth 
marker located outside of the real object; and 

a rendering device for rendering the virtual graphics guide onto a real view 
of the real object and a real Instrument such that the real instrument can be 
inserted to a depth determined in the augmented view by alignment of a 
predetermined real feature of the real instrument with the virtual depth marker, 
the real feature being located along the length of the real instrument at a certain 
distance from the real instrument tip, and remains external to the real object 
during insertion. 

37. The apparatus of claim 36, wherein said virtual graphics guide generator 
and positioner determines an optimal location for the real instrument with respect 
to the real target, and calculates the proximity of said predetermined portion of 
the real instrument to the real target based on the optimal location and the 
geometry of the real instrument. 

38. The apparatus of claim 37, wherein the proximity comprises a range of 
proximities and said virtual graphics guide generator and positioner determines 
an optima! range of locations for the predetermined portion of the real instrument 
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with respect to the real target, and calculates the range of proximities of the 
predetermined portion of the real instrument to the real target based on the 
optimal range and the geometry of the real instrument. 

39. The apparatus of claim 37, wherein the proximity corresponds to a final 
forward position of the predetermined portion of the real instrument with respect 
to the real target. 

40. The apparatus of claim 36, comprising further a display device to display 
the augmented view rendered by the rendering device to the user. 
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